breaking(joex): replace unoconv with unoserver for office conversion - #44
breaking(joex): replace unoconv with unoserver for office conversion#44tiborrr wants to merge 1 commit into
Conversation
Start unoserver in the joex entrypoint (with unoping readiness) and install LibreOffice + unoserver instead of the deprecated unoconv package. Concurrent converts are queued by the server, avoiding LibreOffice listener deadlocks (eikek/docspell#3345). Co-authored-by: Cursor <cursoragent@cursor.com>
|
Companion docspell defaults PR: eikek/docspell#3347 (eikek/docspell#3345) |
| echo "Starting unoserver" | ||
| # Queues office converts so concurrent joex workers do not deadlock LibreOffice. | ||
| # See https://github.com/eikek/docspell/issues/3345 | ||
| unoserver --conversion-timeout 120 & | ||
|
|
||
| echo "Waiting for unoserver to become ready" | ||
| i=0 | ||
| while [ "$i" -lt 60 ]; do | ||
| if unoping >/dev/null 2>&1; then | ||
| echo "unoserver is ready" | ||
| break | ||
| fi | ||
| i=$((i + 1)) | ||
| sleep 1 | ||
| done | ||
|
|
||
| if ! unoping >/dev/null 2>&1; then | ||
| echo "WARNING: unoserver did not become ready within 60s; office conversion may fail" >&2 | ||
| fi |
There was a problem hiding this comment.
I wonder if this whole mess wouldn't better live in joex itself instead of in a random entrypoint script.
There was a problem hiding this comment.
(I also didn't like the previous unoconv -l & line in here)
There was a problem hiding this comment.
I totally agree. Sorry my cursor instance was a bit too quick in filing this PR, but I totally agree here. I will investigate.
There was a problem hiding this comment.
I also think that a sidecar container in kubernetes would work quite well for this, but not sure how to handle non-kubernetes deployments then.
There was a problem hiding this comment.
In docker compose it's also easy. I guess for local deployment peeps just have a single docker unoserver instance running that they can call or install it bare metal I guess and run it in the background.
There was a problem hiding this comment.
I think it's good to seperate it
|
Updated PR to the sidecar architecture (replacing the joex entrypoint approach):
Parallel xlsx verification (5 concurrent converts) succeeded against the sidecar. |
|
@pschichtel if we have a separate Libre Office instance then this will break the instances that run bare metal. We could also just implement the a locking mechanism in joex that only one office document can be processed at a time. That is fine with me, but would like to have your opinion. EDIT: In the refactor I would implement it as a managed daeom in scala |
|
Sounds good to me. So you'd switch to unoserver and manage that process from scala thus also getting rid of the entrypoint script, right? |
2 participants
Summary
unoconvwith LibreOffice +unoserver(pip) in the joex imageunoserver --conversion-timeout 120and waits forunopingbefore launching joexpool-size > 1(eikek/docspell#3345)Breaking
Images that relied on
unoconv -lnow require matching Docspell defaults that callunoconvert(companion PR on eikek/docspell). Release/tag these together.Test plan
images/Dockerfile.joexunopingunoconvertavailable on PATHcc @pschichtel
Made with Cursor